Original Paper: https://arxiv.org/abs/2209.12573, Authors: Sahar Al Ajmi, Khizar Hayat, Alaa M. Al Obaidi, Naresh Kumar, Munaf Najmuldeen (University of Nizwa, Sultanate of Oman)
In the era of rapid advancement of artificial intelligence and digital audio processing technologies has made the creation of faked or mimicked speech not only possible but also increasingly convincing. This development poses significant risks, ranging from personal identity theft to misinformation and security threats. Nowadays when digital communication predominates, the authenticity of audio content has become a critical concern.
This research work introduces a neural network-based method to distinguish real from mimicked speech without relying on any reference or real source audio, termed as 'zero-knowledge' detection.
It leverages a range of sophisticated audio feature extraction techniques, including spectral and cepstral analyses, to train a model capable of identifying faked speech with remarkable accuracy.
The model's ability to operate in a blind mode, without the need for comparison against authentic audio, sets a new precedent in the field of digital audio forensics.
Audio Vs Speech: Audio is a waveform data in which the amplitude changes with respect to time whereas Speech is the oral communication and pertains to the act of speaking and expressing thoughts and emotions by sounds and gestures.
An audio signal is a representation of sound in function to the vibration of sound that is audible to human ear (20Hz to 20kHz). For machines, the processing of audio is different from humans. In order for the machine to get a sound it should have the recorder and then save the sudio in machine processable formats like mp3, WMA,WAV etc.
Tempral and Spectral features: Features from speech signals can be broadly classified as temporal and spectral features. The temporal features are time domain features having simple physical interpretation and easy to compute.
The spectral features, on the other hand, are frequency-based features that are extracted after passing the time domain signal to the frequency domain using Fourier or other similar transforms.
In the context of audio signals such features may be helpful in the identification of pitch, notes, rhythm and melody etc.
Spectrum and cepstrum are two important frequency-based concepts in audio processing:
A spectrum is mathematically a Fourier transform of a signal which converts a time-domain signal into frequency domain, i.e. spectrum is the audio signal in frequency domain.
A cepstrum is the log of the magnitude of the spectrum followed by an inverse Fourier transform. That’s why its domain is neither frequency nor the time; its domain is called quefrency. Cepstrum can be said of as a sequence of numbers that characterize a frame of speech.
The data is collected from a number of social media apps and sites and downloaded the audios which were then edited to conform to the proposed model by limiting it to a maximum duration of 20 seconds in WAV format.
One part of the dataset consists of all English audios (both real and mimicked). The second part of the dataset contains a mix of both English and Arabic audios.
The audio files are named so that the first four characters are digits to represen the index and the fifth character is either ’r’ or ’f’ to label the voice as real or faked, respectively. the raw dataset is available for public on GitHub (https://github.com/SaSs7/Dataset).
Our goal is to blindly identify whether a given voice is mimicked or otherwise. Hence for our experiments, a set of independent real and faked audios was required to create the dataset; real and faked voices uttered independently of what is being said and who said it and independent of the language.
import IPython.display as ipd
# Play the audio file
print("Example Real Audio:")
ipd.Audio("0027r.wav")
Example Real Audio:
print("Example Fake Audio:")
ipd.Audio('0040f.wav')
Example Fake Audio:
The use case of this method concerns the scenario of a complete blindness wherein no prior or side information is available about the speaker.
This work's emphasis is on classifying the speech, as faked, or otherwise, under the assumption that no other recorded voice of the speaker is available; whether genuine or disguised. T
he purported speaker is only represented once in the training data; that too either as real or mimicked, but not both. Potentially, this work's method may be very useful in improving efficiency of many audio processing methods, especially, when applied at the pre-processing stage.
For the training set, it is essential that:
Exactly one sample pertains to a purported speaker.
Either the real or the faked voice of a given speaker is part of the dataset; in fact, they should be mutually exclusive.
The spoken words are not required to be identical.
Keeping the above in view, We extracted our dataset from the raw dataset to contain 933 English-only audio samples and 1127 samples in both English and Arabic.
Zero crossing rate: It indicates the number of times the value of the signal changes between positive and negative and vise versa. It is also used to measure the noise in a signal, and it usually gives high value in case of a noisy signal.
Spectral centroid: It is a feature based on frequency which indicates the location of the center of mass of the spectrum. In audios it is known as a good predictor of “ brightness” of a sound.
Spectral roll off: This feature is used to differentiate between the harmonic sound (below roll off) and the noise sound (above roll off). It is known as the energy spectrum under a specific percentage that is defined by the used (85% by default).
Spectral bandwidth: The difference between the higher and lower frequencies in a group of continuous frequencies.
Chroma: This representation for audio where the spectrum is divided onto 12 bins representing the 12 distinct semitones (or chroma) of the musical octave.
Root Mean Square Energy (RMSE): RMSE represents the energy of the signal, and shows how loud the signal is.
Spectral flux: It measures how quick the power spectrum of a signal is changing, and it is calculated by comparing the changes of the power spectrum between one frame and the frame before it.
Spectral density: It is the measure of signal’s power content against frequency.
Cepstral Features: These are, as stated above, quefrency domain features with the following being considered important:
Mel Frequency Cepstral Coefficients (MFCCs): MFCCs are widely used features for speech recognition. The Mel-frequency scale represents subjective or perceived pitch as its construction is based on pairwise comparisons of sinusoidal tones. The conversion between Hertz (f) and Mel (m) frequencies can be generalized as:
$$ \text{m} = 2595 . \log[1 + \frac{f}{700}],$$ $$ f = 700.(10^{\frac{m}{2595}}-1)$$
MFCCs are obtained by applying a short time Fourier transform to window-based slices from the audio signal, followed by calculating the power spectrum and consequently filter banks (triangular in shape). The filter bank coefficients are highly correlated and one way to de-correlate them is by applying a Discrete Cosine Transform DCT to get a compressed representation in the form of MFCC. Typically, MFCC 2-13 (i.e. 12 coefficients) are kept and the rest are discarded.
Gammatone Frequency Cepstral Coefficients (GFCCs): used in a number of speech processing applications, such as speaker identification. A Gammatone filter bank approximates the impulse response of the auditory nerve fiber thus emulating human hearing and its shape can be likened to a Gamma function $(e^{ −2π(f_{c})bt})$ modulating the tone $(cos(2 \pi f_{c} t + \phi))$: $$g(t) = at^{n-1}e^{2 \pi (f_{c}) b t} cos(2 \pi f_{c} t + \phi)$$
Where $a$ is peak value, $n$ the order of the filter, $b$ the bandwidth, $f_{c}$ the characteristic frequency and $\phi$ is initial phase. $f_{c}$ and $b$ can be derived from Equivalent Rectangular Bandwidth (ERB) scale, using the following equation: $$ \text{ERB}(f_{c}) = 24.7 \cdot (4.37 \cdot \frac{f_{c}}{1000} + 1) $$ $$ b = 1.019 \times \text{ERB}(f_{c})$$
The data is standardized, which means that they will have a mean of 0 and a standard deviation of 1.
Using sklearn.model_selection, the feature set is first partitioned to training and testing sets. During the training phase, the training set is dynamically partitioned to training and validation parts.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm
import librosa
import csv
import os
import pandas as pd
import seaborn as sns
from keras import models
from keras import layers
from datetime import datetime
from keras import backend as K
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.preprocessing.image import ImageDataGenerator
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, roc_curve
from matplotlib import pyplot
C:\Users\shash\Desktop\Environments\deeplearning_class\lib\site-packages\requests\__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (5.1.0)/charset_normalizer (2.0.12) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
'''The following function is to extract the features and store them in the csv file, this function is tested on the
dataset but for the rest of the experiment we will use the pre stored csv files to save time and the uniformity
'''
def extractWavFeatures(soundFilesFolder, csvFileName):
print("The features of the files in the folder "+soundFilesFolder+" will be saved to "+csvFileName)
header = 'filename chroma_stft rmse spectral_centroid spectral_bandwidth rolloff zero_crossing_rate'
for i in range(1, 21):
header += f' mfcc{i}'
header += ' label'
header = header.split()
print('CSV Header: ', header)
file = open(csvFileName, 'w', newline='')
#with file:
writer = csv.writer(file)
writer.writerow(header)
genres = 'real fake'.split()
for filename in os.listdir(soundFilesFolder):
number = f'{soundFilesFolder}/{filename}'
y, sr = librosa.load(number, mono=True, duration=3)
# remove leading and trailing silence
y, index = librosa.effects.trim(y)
chroma_stft = librosa.feature.chroma_stft(y=y, sr=sr)
rmse = librosa.feature.rms(y=y)
spec_cent = librosa.feature.spectral_centroid(y=y, sr=sr)
spec_bw = librosa.feature.spectral_bandwidth(y=y, sr=sr)
rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)
zcr = librosa.feature.zero_crossing_rate(y)
mfcc = librosa.feature.mfcc(y=y, sr=sr)
to_append = f'{filename} {np.mean(chroma_stft)} {np.mean(rmse)} {np.mean(spec_cent)} {np.mean(spec_bw)} {np.mean(rolloff)} {np.mean(zcr)}'
for e in mfcc:
to_append += f' {np.mean(e)}'
writer.writerow(to_append.split())
file.close()
print("End of extractWavFeatures")
# Defines the names of the CSV files
TRAIN_CSV_FILE = "train_E.csv"
TEST_CSV_FILE = "test_E.csv"
def preProcessData(csvFileName):
print(csvFileName+ " will be preprocessed")
data = pd.read_csv(csvFileName)
filenameArray = data['filename']
soundArray = []
for i in range(len(filenameArray)):
sound = filenameArray[i][4]
if sound == "f":
sound = "0"
else:
sound = "1"
soundArray.append(sound)
data['fake(0)/real(1)'] = soundArray
#Dropping unnecessary columns
data = data.drop(['filename'],axis=1)
data = data.drop(['label'],axis=1)
data = data.drop(['chroma_stft'],axis=1)
data.shape
print("Preprocessing is finished")
print(data)
return data
trainData = preProcessData(TRAIN_CSV_FILE)
testData = preProcessData(TEST_CSV_FILE)
# Splitting the dataset into training, validation and testing dataset
from sklearn.model_selection import train_test_split
X = np.array(trainData.iloc[:, :-1], dtype=float)
y = np.array(trainData.iloc[:, -1], dtype=float)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=4)
#print(y)
X_test = np.array(testData.iloc[:, :-1], dtype=float)
y_test = np.array(testData.iloc[:, -1], dtype=float)
print("Y from training data:", y_train.shape)
print("Y from validation data:", y_val.shape)
print("Y from test data:", y_test.shape)
#Normalizing the dataset
scaler = StandardScaler()
X_train = scaler.fit_transform( X_train )
X_val = scaler.transform( X_val )
X_test = scaler.transform( X_test )
print("X from training data", X_train.shape)
print("X from validation data", X_val.shape)
print("X from test data", X_test.shape)
train_E.csv will be preprocessed
Preprocessing is finished
rmse spectral_centroid spectral_bandwidth rolloff \
0 0.034180 1668.499512 1931.042986 3549.831355
1 0.021000 2285.333847 2078.051071 4437.413424
2 0.050395 2520.792678 2095.533805 4588.145846
3 0.033419 1244.333072 1363.402056 2314.322416
4 0.155624 610.774970 1354.636615 1060.675894
... ... ... ... ...
1721 0.048439 2388.029478 2371.731559 4676.266339
1722 0.021571 1974.492901 1784.187394 4006.086801
1723 0.026254 2469.095902 2081.365811 4407.598220
1724 0.086118 2466.251183 1995.966252 4459.277907
1725 0.015375 2224.121561 2600.205771 5107.592961
zero_crossing_rate mfcc1 mfcc2 mfcc3 mfcc4 \
0 0.075883 -317.682739 125.594170 -10.551296 39.107578
1 0.114938 -306.967224 86.352386 -45.065014 13.220994
2 0.165828 -293.701935 93.260078 -21.459564 39.693501
3 0.067939 -347.225647 153.921768 -19.860266 30.877554
4 0.010825 -319.213135 123.525932 17.571198 26.839479
... ... ... ... ... ...
1721 0.131002 -279.913635 79.839561 18.476648 30.238594
1722 0.103842 -328.775848 116.810524 -42.521950 64.042877
1723 0.157918 -362.499268 87.291267 -17.928530 40.048580
1724 0.156182 -276.038422 91.641991 -47.719906 12.318085
1725 0.092514 -387.321045 97.893669 18.178455 18.703951
mfcc5 ... mfcc12 mfcc13 mfcc14 mfcc15 mfcc16 \
0 16.686037 ... 5.968605 3.103173 5.685068 -3.517712 3.200421
1 -27.725321 ... -5.060811 -14.218973 -2.106550 -9.817135 3.340336
2 11.339577 ... 9.603717 -14.536390 -12.303791 5.980660 -5.071312
3 -5.580781 ... -11.630945 -0.307690 -8.779072 -1.268764 -7.597798
4 32.150406 ... 13.857342 5.856003 8.262341 10.047656 7.611200
... ... ... ... ... ... ... ...
1721 -0.304735 ... -6.598882 -8.402861 -4.582676 -5.717072 -4.839275
1722 -26.239555 ... -12.563019 5.768204 -6.546067 3.432244 -8.117812
1723 19.528072 ... 1.921619 -11.375549 -4.797867 5.801071 -4.497665
1724 6.437813 ... 2.254773 -9.910292 -8.141305 6.848052 -4.112238
1725 11.645022 ... 0.604516 -4.727904 1.979092 -2.205940 8.250033
mfcc17 mfcc18 mfcc19 mfcc20 fake(0)/real(1)
0 -8.261959 -5.873692 -5.119587 -5.908155 0
1 -11.947133 -11.330423 -13.046369 -6.897550 0
2 -8.650858 2.234367 -19.276699 8.082246 0
3 -3.591662 -10.366803 -6.630301 -2.690367 1
4 1.222090 9.238463 6.456387 0.924751 0
... ... ... ... ... ...
1721 -12.386527 -6.143230 -7.573254 -5.020414 0
1722 7.497486 -8.267379 -4.207213 -1.210499 1
1723 0.083682 3.859899 -13.733687 4.992855 0
1724 -7.959238 1.330388 -10.561821 0.802223 0
1725 2.387479 0.143632 -4.710601 -1.307774 0
[1726 rows x 26 columns]
test_E.csv will be preprocessed
Preprocessing is finished
rmse spectral_centroid spectral_bandwidth rolloff \
0 0.022751 2249.787185 2376.128899 4695.149301
1 0.049245 3465.137831 2820.354304 6801.179387
2 0.020501 1525.980523 1548.727499 2772.896822
3 0.027919 2323.886833 2466.548211 5144.861967
4 0.052694 2629.982238 2126.740805 5233.562200
.. ... ... ... ...
129 0.004787 1750.094520 2048.088351 3545.441895
130 0.037639 1627.004273 1580.014698 2967.772311
131 0.049074 1622.427320 1529.049987 2990.961914
132 0.020168 2309.825471 2097.182748 4317.572867
133 0.046054 1507.294385 1552.013291 2854.060434
zero_crossing_rate mfcc1 mfcc2 mfcc3 mfcc4 \
0 0.104353 -266.537506 100.207375 -2.550580 26.464508
1 0.205529 -174.206604 42.279720 -6.944875 15.448704
2 0.079714 -381.317749 140.187225 -20.570433 31.324745
3 0.112954 -274.121552 107.466934 19.567888 14.321390
4 0.157039 -277.065918 87.989265 -7.939995 54.299805
.. ... ... ... ... ...
129 0.080123 -449.910583 120.838371 -4.420218 34.616032
130 0.084593 -295.014557 137.690628 -38.313507 24.982122
131 0.102840 -295.802673 137.366684 -27.608025 28.890900
132 0.124921 -387.965210 98.773605 -21.788591 4.870010
133 0.091902 -311.278046 154.806519 -6.527535 25.394995
mfcc5 ... mfcc12 mfcc13 mfcc14 mfcc15 mfcc16 \
0 -2.508199 ... 3.351594 -7.754051 -3.105744 -2.000808 -2.967053
1 3.816530 ... -2.126826 -10.584426 -2.605693 -11.338287 -1.758550
2 -24.516249 ... -2.256354 13.272095 -4.865770 12.484118 -4.466461
3 -16.991381 ... 3.902887 -2.239980 -3.745878 -3.009543 -0.513194
4 -41.605099 ... -5.633950 10.789930 -5.878231 6.949822 -18.838655
.. ... ... ... ... ... ... ...
129 8.832709 ... 2.298683 -2.077702 4.704892 1.814711 3.613112
130 16.592770 ... -6.389162 7.021423 -2.761451 -19.340149 7.219986
131 -20.974297 ... -4.458465 2.848928 -11.792307 7.006453 -9.080160
132 -9.948197 ... -2.269905 -3.676280 -0.809023 -7.236962 2.130877
133 -31.627464 ... -4.106587 1.120149 -11.142779 8.875122 -2.274678
mfcc17 mfcc18 mfcc19 mfcc20 fake(0)/real(1)
0 -6.752093 -0.381953 -9.223604 6.556966 1
1 -10.134315 -1.485182 -6.332491 -3.386185 0
2 3.078789 1.599398 0.315203 -3.342583 1
3 -7.980110 -0.782919 -6.861320 1.792280 1
4 2.983628 -11.509598 -4.806178 -11.076943 1
.. ... ... ... ... ...
129 -1.539815 4.534101 -2.730610 0.830440 1
130 -7.655596 -9.247864 -1.133270 -1.394202 0
131 2.900971 -2.835001 -2.490820 -2.959754 1
132 -8.827484 1.049352 -6.226123 -8.802943 1
133 -3.146388 -3.810700 0.087730 -3.284060 1
[134 rows x 26 columns]
Y from training data: (1380,)
Y from validation data: (346,)
Y from test data: (134,)
X from training data (1380, 25)
X from validation data (346, 25)
X from test data (134, 25)
#Creating a Model
# model 1
model = models.Sequential()
model.add(layers.Dense(256, activation='relu', input_shape=(X_train.shape[1],)))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(2, activation='softmax'))
# Learning Process of a model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 256) 6656
dropout (Dropout) (None, 256) 0
dense_1 (Dense) (None, 128) 32896
dropout_1 (Dropout) (None, 128) 0
dropout_2 (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
dropout_3 (Dropout) (None, 64) 0
dense_3 (Dense) (None, 2) 130
=================================================================
Total params: 47,938
Trainable params: 47,938
Non-trainable params: 0
_________________________________________________________________
# Calculate pre-training accuracy
score = model.evaluate(X_test, y_test, verbose=1)
accuracy = 100*score[1]
print("Pre-training accuracy: %.4f%%" % accuracy)
5/5 [==============================] - 1s 5ms/step - loss: 0.7151 - accuracy: 0.5075 Pre-training accuracy: 50.7463%
start = datetime.now()
#training
history = model.fit(X_train,
y_train,
validation_split=(0.1),
epochs=135,
batch_size=128)
duration = datetime.now() - start
print("Training completed in time: ", duration)
Epoch 1/135 10/10 [==============================] - 1s 17ms/step - loss: 0.7516 - accuracy: 0.5902 - val_loss: 0.3835 - val_accuracy: 0.9348 Epoch 2/135 10/10 [==============================] - 0s 12ms/step - loss: 0.4981 - accuracy: 0.7544 - val_loss: 0.2365 - val_accuracy: 0.9420 Epoch 3/135 10/10 [==============================] - 0s 11ms/step - loss: 0.3563 - accuracy: 0.8486 - val_loss: 0.1714 - val_accuracy: 0.9348 Epoch 4/135 10/10 [==============================] - 0s 11ms/step - loss: 0.2800 - accuracy: 0.8857 - val_loss: 0.1505 - val_accuracy: 0.9348 Epoch 5/135 10/10 [==============================] - 0s 13ms/step - loss: 0.2623 - accuracy: 0.9090 - val_loss: 0.1420 - val_accuracy: 0.9420 Epoch 6/135 10/10 [==============================] - 0s 14ms/step - loss: 0.2383 - accuracy: 0.9130 - val_loss: 0.1367 - val_accuracy: 0.9420 Epoch 7/135 10/10 [==============================] - 0s 11ms/step - loss: 0.2091 - accuracy: 0.9130 - val_loss: 0.1329 - val_accuracy: 0.9420 Epoch 8/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1841 - accuracy: 0.9275 - val_loss: 0.1292 - val_accuracy: 0.9493 Epoch 9/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1726 - accuracy: 0.9308 - val_loss: 0.1272 - val_accuracy: 0.9493 Epoch 10/135 10/10 [==============================] - 0s 14ms/step - loss: 0.1846 - accuracy: 0.9211 - val_loss: 0.1230 - val_accuracy: 0.9638 Epoch 11/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1654 - accuracy: 0.9324 - val_loss: 0.1211 - val_accuracy: 0.9638 Epoch 12/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1695 - accuracy: 0.9380 - val_loss: 0.1171 - val_accuracy: 0.9710 Epoch 13/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1498 - accuracy: 0.9420 - val_loss: 0.1142 - val_accuracy: 0.9638 Epoch 14/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1491 - accuracy: 0.9380 - val_loss: 0.1108 - val_accuracy: 0.9638 Epoch 15/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1483 - accuracy: 0.9388 - val_loss: 0.1102 - val_accuracy: 0.9638 Epoch 16/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1339 - accuracy: 0.9517 - val_loss: 0.1091 - val_accuracy: 0.9638 Epoch 17/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1346 - accuracy: 0.9412 - val_loss: 0.1047 - val_accuracy: 0.9710 Epoch 18/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1311 - accuracy: 0.9469 - val_loss: 0.0996 - val_accuracy: 0.9710 Epoch 19/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1259 - accuracy: 0.9525 - val_loss: 0.0970 - val_accuracy: 0.9710 Epoch 20/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1131 - accuracy: 0.9541 - val_loss: 0.0942 - val_accuracy: 0.9710 Epoch 21/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1215 - accuracy: 0.9485 - val_loss: 0.0929 - val_accuracy: 0.9710 Epoch 22/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1125 - accuracy: 0.9557 - val_loss: 0.0908 - val_accuracy: 0.9710 Epoch 23/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1056 - accuracy: 0.9525 - val_loss: 0.0894 - val_accuracy: 0.9638 Epoch 24/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1170 - accuracy: 0.9525 - val_loss: 0.0879 - val_accuracy: 0.9638 Epoch 25/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0981 - accuracy: 0.9533 - val_loss: 0.0866 - val_accuracy: 0.9710 Epoch 26/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1047 - accuracy: 0.9565 - val_loss: 0.0849 - val_accuracy: 0.9710 Epoch 27/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1073 - accuracy: 0.9557 - val_loss: 0.0845 - val_accuracy: 0.9710 Epoch 28/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0820 - accuracy: 0.9654 - val_loss: 0.0827 - val_accuracy: 0.9710 Epoch 29/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0917 - accuracy: 0.9630 - val_loss: 0.0811 - val_accuracy: 0.9710 Epoch 30/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0857 - accuracy: 0.9670 - val_loss: 0.0806 - val_accuracy: 0.9783 Epoch 31/135 10/10 [==============================] - 0s 16ms/step - loss: 0.0923 - accuracy: 0.9686 - val_loss: 0.0781 - val_accuracy: 0.9855 Epoch 32/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0917 - accuracy: 0.9565 - val_loss: 0.0778 - val_accuracy: 0.9855 Epoch 33/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0986 - accuracy: 0.9597 - val_loss: 0.0781 - val_accuracy: 0.9855 Epoch 34/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0823 - accuracy: 0.9654 - val_loss: 0.0775 - val_accuracy: 0.9855 Epoch 35/135 10/10 [==============================] - 0s 14ms/step - loss: 0.0809 - accuracy: 0.9638 - val_loss: 0.0765 - val_accuracy: 0.9855 Epoch 36/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0821 - accuracy: 0.9710 - val_loss: 0.0753 - val_accuracy: 0.9855 Epoch 37/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0935 - accuracy: 0.9614 - val_loss: 0.0754 - val_accuracy: 0.9783 Epoch 38/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0705 - accuracy: 0.9750 - val_loss: 0.0729 - val_accuracy: 0.9783 Epoch 39/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0853 - accuracy: 0.9646 - val_loss: 0.0723 - val_accuracy: 0.9783 Epoch 40/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0803 - accuracy: 0.9646 - val_loss: 0.0728 - val_accuracy: 0.9855 Epoch 41/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0762 - accuracy: 0.9662 - val_loss: 0.0753 - val_accuracy: 0.9783 Epoch 42/135 10/10 [==============================] - 0s 14ms/step - loss: 0.0809 - accuracy: 0.9718 - val_loss: 0.0742 - val_accuracy: 0.9783 Epoch 43/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0637 - accuracy: 0.9726 - val_loss: 0.0737 - val_accuracy: 0.9783 Epoch 44/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0723 - accuracy: 0.9678 - val_loss: 0.0737 - val_accuracy: 0.9783 Epoch 45/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0618 - accuracy: 0.9670 - val_loss: 0.0736 - val_accuracy: 0.9783 Epoch 46/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0639 - accuracy: 0.9799 - val_loss: 0.0722 - val_accuracy: 0.9783 Epoch 47/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0570 - accuracy: 0.9831 - val_loss: 0.0707 - val_accuracy: 0.9783 Epoch 48/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0606 - accuracy: 0.9710 - val_loss: 0.0726 - val_accuracy: 0.9855 Epoch 49/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0717 - accuracy: 0.9767 - val_loss: 0.0734 - val_accuracy: 0.9783 Epoch 50/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0583 - accuracy: 0.9791 - val_loss: 0.0737 - val_accuracy: 0.9783 Epoch 51/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0614 - accuracy: 0.9742 - val_loss: 0.0726 - val_accuracy: 0.9783 Epoch 52/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0645 - accuracy: 0.9742 - val_loss: 0.0731 - val_accuracy: 0.9783 Epoch 53/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0577 - accuracy: 0.9767 - val_loss: 0.0745 - val_accuracy: 0.9783 Epoch 54/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0532 - accuracy: 0.9823 - val_loss: 0.0756 - val_accuracy: 0.9783 Epoch 55/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0643 - accuracy: 0.9783 - val_loss: 0.0750 - val_accuracy: 0.9710 Epoch 56/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0499 - accuracy: 0.9807 - val_loss: 0.0766 - val_accuracy: 0.9710 Epoch 57/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0459 - accuracy: 0.9799 - val_loss: 0.0808 - val_accuracy: 0.9783 Epoch 58/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0525 - accuracy: 0.9823 - val_loss: 0.0796 - val_accuracy: 0.9783 Epoch 59/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0520 - accuracy: 0.9823 - val_loss: 0.0777 - val_accuracy: 0.9783 Epoch 60/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0450 - accuracy: 0.9823 - val_loss: 0.0809 - val_accuracy: 0.9710 Epoch 61/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0500 - accuracy: 0.9799 - val_loss: 0.0844 - val_accuracy: 0.9710 Epoch 62/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0480 - accuracy: 0.9815 - val_loss: 0.0813 - val_accuracy: 0.9710 Epoch 63/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0581 - accuracy: 0.9783 - val_loss: 0.0805 - val_accuracy: 0.9710 Epoch 64/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0529 - accuracy: 0.9815 - val_loss: 0.0792 - val_accuracy: 0.9783 Epoch 65/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0515 - accuracy: 0.9823 - val_loss: 0.0768 - val_accuracy: 0.9710 Epoch 66/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0518 - accuracy: 0.9742 - val_loss: 0.0784 - val_accuracy: 0.9710 Epoch 67/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0387 - accuracy: 0.9871 - val_loss: 0.0801 - val_accuracy: 0.9710 Epoch 68/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0331 - accuracy: 0.9903 - val_loss: 0.0826 - val_accuracy: 0.9855 Epoch 69/135 10/10 [==============================] - 0s 17ms/step - loss: 0.0380 - accuracy: 0.9871 - val_loss: 0.0843 - val_accuracy: 0.9783 Epoch 70/135 10/10 [==============================] - 0s 16ms/step - loss: 0.0522 - accuracy: 0.9839 - val_loss: 0.0853 - val_accuracy: 0.9710 Epoch 71/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0470 - accuracy: 0.9823 - val_loss: 0.0868 - val_accuracy: 0.9783 Epoch 72/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0461 - accuracy: 0.9807 - val_loss: 0.0910 - val_accuracy: 0.9783 Epoch 73/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0418 - accuracy: 0.9855 - val_loss: 0.0897 - val_accuracy: 0.9783 Epoch 74/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0472 - accuracy: 0.9823 - val_loss: 0.0878 - val_accuracy: 0.9855 Epoch 75/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0373 - accuracy: 0.9831 - val_loss: 0.0876 - val_accuracy: 0.9855 Epoch 76/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0373 - accuracy: 0.9839 - val_loss: 0.0921 - val_accuracy: 0.9783 Epoch 77/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0278 - accuracy: 0.9903 - val_loss: 0.0936 - val_accuracy: 0.9783 Epoch 78/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0301 - accuracy: 0.9928 - val_loss: 0.0928 - val_accuracy: 0.9783 Epoch 79/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0375 - accuracy: 0.9847 - val_loss: 0.0910 - val_accuracy: 0.9855 Epoch 80/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0246 - accuracy: 0.9911 - val_loss: 0.0921 - val_accuracy: 0.9783 Epoch 81/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0203 - accuracy: 0.9944 - val_loss: 0.0949 - val_accuracy: 0.9783 Epoch 82/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0233 - accuracy: 0.9919 - val_loss: 0.0965 - val_accuracy: 0.9783 Epoch 83/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0262 - accuracy: 0.9911 - val_loss: 0.1039 - val_accuracy: 0.9783 Epoch 84/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0319 - accuracy: 0.9895 - val_loss: 0.1089 - val_accuracy: 0.9783 Epoch 85/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0299 - accuracy: 0.9911 - val_loss: 0.1079 - val_accuracy: 0.9710 Epoch 86/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0208 - accuracy: 0.9952 - val_loss: 0.1092 - val_accuracy: 0.9710 Epoch 87/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0357 - accuracy: 0.9895 - val_loss: 0.1108 - val_accuracy: 0.9783 Epoch 88/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0321 - accuracy: 0.9863 - val_loss: 0.1167 - val_accuracy: 0.9783 Epoch 89/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0355 - accuracy: 0.9895 - val_loss: 0.1132 - val_accuracy: 0.9783 Epoch 90/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0276 - accuracy: 0.9903 - val_loss: 0.1100 - val_accuracy: 0.9783 Epoch 91/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0193 - accuracy: 0.9919 - val_loss: 0.1141 - val_accuracy: 0.9710 Epoch 92/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0257 - accuracy: 0.9919 - val_loss: 0.1152 - val_accuracy: 0.9783 Epoch 93/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0305 - accuracy: 0.9887 - val_loss: 0.1103 - val_accuracy: 0.9638 Epoch 94/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0233 - accuracy: 0.9887 - val_loss: 0.1076 - val_accuracy: 0.9710 Epoch 95/135 10/10 [==============================] - 0s 17ms/step - loss: 0.0242 - accuracy: 0.9887 - val_loss: 0.1040 - val_accuracy: 0.9710 Epoch 96/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0212 - accuracy: 0.9911 - val_loss: 0.1086 - val_accuracy: 0.9710 Epoch 97/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0236 - accuracy: 0.9928 - val_loss: 0.1127 - val_accuracy: 0.9710 Epoch 98/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0262 - accuracy: 0.9911 - val_loss: 0.1039 - val_accuracy: 0.9783 Epoch 99/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0287 - accuracy: 0.9895 - val_loss: 0.1013 - val_accuracy: 0.9783 Epoch 100/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0378 - accuracy: 0.9871 - val_loss: 0.1016 - val_accuracy: 0.9783 Epoch 101/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0310 - accuracy: 0.9911 - val_loss: 0.1082 - val_accuracy: 0.9710 Epoch 102/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0243 - accuracy: 0.9895 - val_loss: 0.1056 - val_accuracy: 0.9710 Epoch 103/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0203 - accuracy: 0.9928 - val_loss: 0.1049 - val_accuracy: 0.9710 Epoch 104/135 10/10 [==============================] - 0s 14ms/step - loss: 0.0322 - accuracy: 0.9887 - val_loss: 0.1020 - val_accuracy: 0.9710 Epoch 105/135 10/10 [==============================] - 0s 19ms/step - loss: 0.0272 - accuracy: 0.9887 - val_loss: 0.1008 - val_accuracy: 0.9710 Epoch 106/135 10/10 [==============================] - 0s 19ms/step - loss: 0.0205 - accuracy: 0.9919 - val_loss: 0.1070 - val_accuracy: 0.9710 Epoch 107/135 10/10 [==============================] - 0s 14ms/step - loss: 0.0255 - accuracy: 0.9952 - val_loss: 0.1173 - val_accuracy: 0.9710 Epoch 108/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0180 - accuracy: 0.9960 - val_loss: 0.1193 - val_accuracy: 0.9710 Epoch 109/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0168 - accuracy: 0.9936 - val_loss: 0.1189 - val_accuracy: 0.9710 Epoch 110/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0267 - accuracy: 0.9928 - val_loss: 0.1175 - val_accuracy: 0.9710 Epoch 111/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0168 - accuracy: 0.9928 - val_loss: 0.1155 - val_accuracy: 0.9710 Epoch 112/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0189 - accuracy: 0.9911 - val_loss: 0.1209 - val_accuracy: 0.9710 Epoch 113/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0197 - accuracy: 0.9919 - val_loss: 0.1206 - val_accuracy: 0.9638 Epoch 114/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0218 - accuracy: 0.9952 - val_loss: 0.1275 - val_accuracy: 0.9638 Epoch 115/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0132 - accuracy: 0.9984 - val_loss: 0.1288 - val_accuracy: 0.9638 Epoch 116/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0169 - accuracy: 0.9960 - val_loss: 0.1295 - val_accuracy: 0.9638 Epoch 117/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0245 - accuracy: 0.9936 - val_loss: 0.1324 - val_accuracy: 0.9565 Epoch 118/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0112 - accuracy: 0.9976 - val_loss: 0.1379 - val_accuracy: 0.9638 Epoch 119/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0132 - accuracy: 0.9968 - val_loss: 0.1450 - val_accuracy: 0.9565 Epoch 120/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0119 - accuracy: 0.9952 - val_loss: 0.1601 - val_accuracy: 0.9565 Epoch 121/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0114 - accuracy: 0.9960 - val_loss: 0.1712 - val_accuracy: 0.9565 Epoch 122/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0239 - accuracy: 0.9895 - val_loss: 0.1563 - val_accuracy: 0.9565 Epoch 123/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0211 - accuracy: 0.9903 - val_loss: 0.1400 - val_accuracy: 0.9710 Epoch 124/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0145 - accuracy: 0.9952 - val_loss: 0.1433 - val_accuracy: 0.9710 Epoch 125/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0098 - accuracy: 0.9984 - val_loss: 0.1475 - val_accuracy: 0.9710 Epoch 126/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0183 - accuracy: 0.9944 - val_loss: 0.1513 - val_accuracy: 0.9710 Epoch 127/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0169 - accuracy: 0.9944 - val_loss: 0.1559 - val_accuracy: 0.9710 Epoch 128/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0120 - accuracy: 0.9968 - val_loss: 0.1620 - val_accuracy: 0.9710 Epoch 129/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0128 - accuracy: 0.9944 - val_loss: 0.1594 - val_accuracy: 0.9710 Epoch 130/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0103 - accuracy: 0.9984 - val_loss: 0.1602 - val_accuracy: 0.9710 Epoch 131/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0121 - accuracy: 0.9952 - val_loss: 0.1616 - val_accuracy: 0.9638 Epoch 132/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0202 - accuracy: 0.9928 - val_loss: 0.1412 - val_accuracy: 0.9565 Epoch 133/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0138 - accuracy: 0.9944 - val_loss: 0.1327 - val_accuracy: 0.9638 Epoch 134/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0166 - accuracy: 0.9960 - val_loss: 0.1290 - val_accuracy: 0.9710 Epoch 135/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0145 - accuracy: 0.9936 - val_loss: 0.1429 - val_accuracy: 0.9638 Training completed in time: 0:00:16.783809
score = model.evaluate(X_train, y_train, verbose=0)
print("Training Accuracy: ", score[1])
score = model.evaluate(X_test, y_test, verbose=0)
print("Testing Accuracy: ", score[1])
Training Accuracy: 0.9963768124580383 Testing Accuracy: 0.9328358173370361
# plot training history
from matplotlib import pyplot
pyplot.plot(history.history['loss'], label='train')
pyplot.plot(history.history['val_loss'], label='val')
pyplot.legend()
pyplot.title('Loss Plot')
pyplot.show()
plt.plot(history.history['accuracy'], label='train acc')
plt.plot(history.history['val_accuracy'], label='val acc')
plt.legend()
pyplot.title('Accuracy Plot')
plt.show()
plt.savefig('AccVal_acc')
<Figure size 432x288 with 0 Axes>
def report(X_data, y_data):
#Confution Matrix and Classification Report
Y_pred = np.argmax(model.predict(X_data), axis=-1)
y_test_num = y_data.astype(np.int64)
conf_mt = confusion_matrix(y_test_num, Y_pred)
ax = sns.heatmap(conf_mt, annot=True, cmap='Blues')
ax.set_title('Seaborn Confusion Matrix with labels\n\n');
ax.set_xlabel('\nPredicted Values')
ax.set_ylabel('Actual Values ');
## Ticket labels - List must be in alphabetical order
ax.xaxis.set_ticklabels(['False','True'])
ax.yaxis.set_ticklabels(['False','True'])
## Display the visualization of the Confusion Matrix.
plt.show()
print('\nClassification Report')
target_names = ["real", "fake"]
print(classification_report(y_test_num, Y_pred))
print("Classification Report for Test Data\n")
report(X_test, y_test)
Classification Report for Test Data
Classification Report
precision recall f1-score support
0 0.88 0.98 0.93 60
1 0.99 0.89 0.94 74
accuracy 0.93 134
macro avg 0.93 0.94 0.93 134
weighted avg 0.94 0.93 0.93 134
#extractWavFeatures("data/arabic_only", 'arabic_only.csv')
arabic_only_data = preProcessData('arabic_only.csv')
X_arabic_only = np.array(arabic_only_data.iloc[:, :-1], dtype=float)
y_arabic_only = np.array(arabic_only_data.iloc[:, -1], dtype=float)
score = model.evaluate(X_arabic_only, y_arabic_only, verbose=0)
print("English model performance Accuracy on arabic audios: ", score[1])
arabic_only.csv will be preprocessed
Preprocessing is finished
rmse spectral_centroid spectral_bandwidth rolloff \
0 0.378636 1775.428684 1859.840679 3144.178936
1 0.282145 2252.033896 2170.612940 4277.239521
2 0.305489 1971.192996 2014.879602 3641.016921
3 0.316786 1900.684679 1970.851634 3573.896484
4 0.309140 1925.626976 2011.648934 3615.010705
.. ... ... ... ...
361 0.032539 1228.649694 1107.885710 2154.582024
362 0.037689 1370.739366 1092.851283 2370.971304
363 0.025142 1536.837631 1684.140324 3028.976300
364 0.030934 1612.977296 1453.964337 3025.746319
365 0.040843 1421.645269 1236.050288 2706.392353
zero_crossing_rate mfcc1 mfcc2 mfcc3 mfcc4 \
0 0.095944 24.454071 125.963654 -39.534588 -0.307133
1 0.136723 16.222038 109.421410 -29.185638 -3.749355
2 0.107988 11.964952 116.451218 -33.871670 0.692575
3 0.098619 13.750207 118.561989 -34.129040 0.080792
4 0.100274 11.485873 115.744904 -29.410231 12.076959
.. ... ... ... ... ...
361 0.102741 -457.476562 121.981621 -19.747944 12.313033
362 0.104383 -415.229828 122.096619 -29.464462 18.472731
363 0.077092 -452.619843 84.652473 3.956119 22.619265
364 0.154503 -461.398529 107.561806 -12.403620 -1.581788
365 0.082749 -395.315674 104.972870 -35.952358 43.691143
mfcc5 ... mfcc12 mfcc13 mfcc14 mfcc15 mfcc16 \
0 -18.157377 ... 2.968295 -3.053141 -2.763731 -1.392925 7.387962
1 -19.295631 ... 5.612494 -0.967798 0.019961 -1.777691 8.309708
2 -17.859730 ... 2.089557 -4.586041 -4.747462 -1.792464 7.347414
3 -24.793882 ... 1.480854 -4.330481 -1.992103 -3.194457 7.737720
4 -18.024185 ... 8.048415 0.080216 -0.360923 -3.936560 5.524590
.. ... ... ... ... ... ... ...
361 0.148104 ... -10.623080 -9.492170 -9.025165 -4.477253 -15.113188
362 0.203553 ... -12.281847 -9.383650 -10.575407 -4.558228 -15.053144
363 -18.121033 ... -14.546600 -7.614517 -9.247508 -5.901526 -6.477323
364 -9.695493 ... -14.577898 -6.021561 -6.558615 -4.738408 -10.026250
365 -1.071144 ... -5.936321 -10.136777 -13.149978 -2.438977 -15.295250
mfcc17 mfcc18 mfcc19 mfcc20 fake(0)/real(1)
0 3.446643 12.743391 10.984439 5.856971 0
1 2.075918 8.519743 9.655573 2.238568 0
2 2.717303 9.987486 11.872904 4.044930 0
3 4.555403 12.709827 12.526717 4.767708 0
4 0.503979 13.820795 11.918027 1.899997 0
.. ... ... ... ... ...
361 -8.034807 -1.690464 -10.685629 -2.954026 1
362 -11.685537 -3.494714 -12.315552 -7.136920 1
363 -1.581654 1.339201 -7.941329 -3.817668 1
364 -5.759439 -1.023010 -7.377100 -7.085113 1
365 -6.321362 -0.496254 -15.105902 -6.497343 1
[366 rows x 26 columns]
English model performance Accuracy on arabic audios: 0.5300546288490295
TRAIN_CSV_FILE_MIX = "train_mix.csv"
TEST_CSV_FILE_MIX = "test_mix.csv"
trainData_MIX = preProcessData(TRAIN_CSV_FILE_MIX)
testData_MIX = preProcessData(TEST_CSV_FILE_MIX)
# Splitting the dataset into training, validation and testing dataset
from sklearn.model_selection import train_test_split
X_mix = np.array(trainData_MIX.iloc[:, :-1], dtype=float)
y_mix = np.array(trainData_MIX.iloc[:, -1], dtype=float)
X_train_mix, X_val_mix, y_train_mix, y_val_mix = train_test_split(X, y, test_size=0.2, random_state=4)
#print(y)
X_test_mix = np.array(testData_MIX.iloc[:, :-1], dtype=float)
y_test_mix = np.array(testData_MIX.iloc[:, -1], dtype=float)
print("Y from training data:", y_train_mix.shape)
print("Y from validation data:", y_val_mix.shape)
print("Y from test data:", y_test_mix.shape)
#Normalizing the dataset
scaler = StandardScaler()
X_train_mix = scaler.fit_transform( X_train_mix )
X_val_mix = scaler.transform( X_val_mix )
X_test_mix = scaler.transform( X_test_mix )
train_mix.csv will be preprocessed
Preprocessing is finished
rmse spectral_centroid spectral_bandwidth rolloff \
0 0.026089 1631.391348 2048.803882 3303.773099
1 0.149174 2297.421531 2779.812643 5302.799730
2 0.021376 2843.856763 2058.380855 5410.962665
3 0.036125 2205.408168 2143.451236 4195.496169
4 0.017126 3002.692762 2708.508928 5999.812951
... ... ... ... ...
1827 0.150160 2662.907157 1943.178048 4488.513371
1828 0.022921 1720.046679 2240.861763 3635.053148
1829 0.060629 1780.883235 1660.125177 3164.552659
1830 0.024021 1893.898462 1850.908678 3928.318810
1831 0.061028 3303.279910 2685.840721 6441.574895
zero_crossing_rate mfcc1 mfcc2 mfcc3 mfcc4 \
0 0.055897 -357.834015 114.628395 -3.186949 23.541771
1 0.081535 -166.742722 83.738495 36.743710 11.386289
2 0.201705 -345.359161 83.147507 -37.619396 35.414497
3 0.110423 -337.024353 83.927864 -6.709937 36.673985
4 0.159213 -364.517090 68.346886 13.572990 18.934473
... ... ... ... ... ...
1827 0.175988 -133.890656 43.262955 -79.595901 10.847193
1828 0.070170 -376.848969 114.303352 17.528410 17.520994
1829 0.128996 -299.044922 132.135422 -15.403905 10.348984
1830 0.096030 -319.315796 122.128395 -32.028782 48.777805
1831 0.201097 -187.362854 50.722042 -2.457825 27.028816
mfcc5 ... mfcc12 mfcc13 mfcc14 mfcc15 mfcc16 \
0 10.463376 ... 8.423969 2.395511 2.702044 2.834919 7.483851
1 16.039284 ... -0.389156 -5.031785 2.839089 -0.938273 -7.384211
2 -41.686028 ... -9.623581 5.395470 -5.730428 9.778461 -12.117192
3 14.218342 ... 4.398200 1.778909 0.857956 -6.648224 4.044764
4 -7.175009 ... 1.764654 -9.753967 5.826037 -1.313259 0.267051
... ... ... ... ... ... ... ...
1827 -11.963476 ... -4.665625 -9.166707 0.265335 10.989515 5.353004
1828 1.260717 ... -3.034056 -5.181716 -3.379348 -7.106627 -0.557103
1829 -28.931149 ... -5.261015 -2.704779 -16.781631 -1.231559 -21.967545
1830 -20.179951 ... -9.990208 -2.386103 -20.630405 0.020682 -14.963440
1831 17.330750 ... 4.443331 1.209611 3.037515 -3.300395 2.490600
mfcc17 mfcc18 mfcc19 mfcc20 fake(0)/real(1)
0 -3.031116 -1.046314 0.167694 1.189663 0
1 -8.164545 6.206331 -2.926403 5.864541 0
2 6.312552 -9.734758 -0.363713 -2.292386 1
3 -13.255052 -8.571074 -4.139808 -1.048170 0
4 -0.483254 1.339024 -5.205956 2.862189 1
... ... ... ... ... ...
1827 -1.554883 7.503642 -18.307858 9.979580 0
1828 -7.136498 2.531166 -1.977663 1.930692 1
1829 -1.443609 -8.314674 -1.335841 -4.335426 1
1830 0.784167 -5.422856 -1.599851 -1.672773 1
1831 -1.907393 2.692633 -1.294818 -7.612365 0
[1832 rows x 26 columns]
test_mix.csv will be preprocessed
Preprocessing is finished
rmse spectral_centroid spectral_bandwidth rolloff \
0 0.056217 2522.439678 2007.515222 4494.145132
1 0.051648 2574.179636 2265.744180 4925.388935
2 0.019592 2343.321058 2482.986345 4678.502479
3 0.013287 2887.820436 2675.220047 5942.584322
4 0.115803 1542.719145 1307.081338 2625.560021
.. ... ... ... ...
187 0.068033 1868.160844 1744.646164 3686.650015
188 0.013315 1800.104228 1622.207467 3554.552096
189 0.078339 1786.284788 1721.544368 3410.693735
190 0.039121 1512.890618 1348.055849 2600.134277
191 0.072058 2145.571726 1843.014767 4117.615667
zero_crossing_rate mfcc1 mfcc2 mfcc3 mfcc4 \
0 0.153929 -268.186493 94.037758 -35.054951 26.919014
1 0.152520 -153.383179 81.967545 -35.802330 14.997083
2 0.165294 -474.756897 58.064846 2.860619 10.119847
3 0.192691 -544.086792 59.993488 29.092371 -8.765768
4 0.098118 -324.529449 107.583191 25.439896 14.684494
.. ... ... ... ... ...
187 0.104518 -195.795731 135.296936 -52.595036 47.155159
188 0.111076 -402.368622 113.297035 -34.362671 34.292114
189 0.090370 -189.456696 137.890778 -45.384018 41.553719
190 0.090370 -356.704254 111.196289 -35.487511 10.343122
191 0.134245 -210.560928 124.955429 -41.691303 43.462074
mfcc5 ... mfcc12 mfcc13 mfcc14 mfcc15 mfcc16 \
0 5.917642 ... 3.857020 -10.830110 -5.273628 1.054516 -0.542576
1 -10.506969 ... 1.391137 -8.231859 -5.799623 -10.824430 -3.988652
2 14.436151 ... 5.970120 -2.222342 -4.197543 -0.545212 -1.104004
3 -34.941689 ... -5.717347 -11.090538 -8.429750 -14.575583 -1.925327
4 -5.129196 ... -2.290231 -4.009184 -11.149485 -2.367398 -4.708204
.. ... ... ... ... ... ... ...
187 -19.302811 ... -10.151624 0.354040 -11.803023 4.766759 -4.466513
188 -23.614893 ... -6.544881 -4.645352 -13.435349 -4.052658 -1.270053
189 -22.338465 ... -8.887204 -0.385658 -10.098969 8.400770 -4.224036
190 -19.749895 ... -1.095852 0.912711 -6.893618 -5.815292 -4.305375
191 -12.299859 ... -6.453499 7.039301 -9.205093 2.205765 -7.330601
mfcc17 mfcc18 mfcc19 mfcc20 fake(0)/real(1)
0 -0.028192 -4.271140 -10.957672 1.104623 0
1 -9.119308 -5.641050 -7.800853 -0.919380 1
2 -5.783376 -2.646659 -0.578387 -3.140889 0
3 -7.216211 0.361122 -2.936738 -1.379924 1
4 -14.604149 -13.166643 -9.039810 -6.038741 0
.. ... ... ... ... ...
187 6.561789 -10.262828 0.855388 -5.535003 0
188 -8.111988 -1.398235 -2.501163 -7.131615 1
189 9.901284 -9.839184 1.546390 -7.532935 0
190 -6.912731 -0.489506 -2.304261 -3.228322 1
191 4.745310 -10.966357 5.639626 -9.209676 0
[192 rows x 26 columns]
Y from training data: (1380,)
Y from validation data: (346,)
Y from test data: (192,)
#Creating a Model
# model
model = models.Sequential()
model.add(layers.Dense(256, activation='relu', input_shape=(X_train_mix.shape[1],)))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(2, activation='softmax'))
# Learning Process of a model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Calculate pre-training accuracy
score = model.evaluate(X_test_mix, y_test_mix, verbose=1)
accuracy = 100*score[1]
print("Pre-training accuracy: %.4f%%" % accuracy)
6/6 [==============================] - 0s 3ms/step - loss: 0.7087 - accuracy: 0.5260 Pre-training accuracy: 52.6042%
start = datetime.now()
#training
history = model.fit(X_train_mix,
y_train_mix,
validation_split=(0.1),
epochs=135,
batch_size=128)
duration = datetime.now() - start
print("Training completed in time: ", duration)
Epoch 1/135 10/10 [==============================] - 1s 25ms/step - loss: 0.7493 - accuracy: 0.5757 - val_loss: 0.4768 - val_accuracy: 0.9130 Epoch 2/135 10/10 [==============================] - 0s 8ms/step - loss: 0.5446 - accuracy: 0.7367 - val_loss: 0.3229 - val_accuracy: 0.9275 Epoch 3/135 10/10 [==============================] - 0s 7ms/step - loss: 0.3898 - accuracy: 0.8438 - val_loss: 0.2156 - val_accuracy: 0.9420 Epoch 4/135 10/10 [==============================] - 0s 9ms/step - loss: 0.2770 - accuracy: 0.8808 - val_loss: 0.1680 - val_accuracy: 0.9420 Epoch 5/135 10/10 [==============================] - 0s 9ms/step - loss: 0.2708 - accuracy: 0.8969 - val_loss: 0.1535 - val_accuracy: 0.9420 Epoch 6/135 10/10 [==============================] - 0s 9ms/step - loss: 0.2554 - accuracy: 0.9034 - val_loss: 0.1465 - val_accuracy: 0.9420 Epoch 7/135 10/10 [==============================] - 0s 10ms/step - loss: 0.2108 - accuracy: 0.9211 - val_loss: 0.1409 - val_accuracy: 0.9420 Epoch 8/135 10/10 [==============================] - 0s 14ms/step - loss: 0.2052 - accuracy: 0.9235 - val_loss: 0.1376 - val_accuracy: 0.9348 Epoch 9/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1759 - accuracy: 0.9316 - val_loss: 0.1338 - val_accuracy: 0.9493 Epoch 10/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1709 - accuracy: 0.9275 - val_loss: 0.1295 - val_accuracy: 0.9493 Epoch 11/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1630 - accuracy: 0.9404 - val_loss: 0.1258 - val_accuracy: 0.9565 Epoch 12/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1709 - accuracy: 0.9316 - val_loss: 0.1196 - val_accuracy: 0.9638 Epoch 13/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1534 - accuracy: 0.9396 - val_loss: 0.1154 - val_accuracy: 0.9638 Epoch 14/135 10/10 [==============================] - 0s 12ms/step - loss: 0.1358 - accuracy: 0.9444 - val_loss: 0.1101 - val_accuracy: 0.9638 Epoch 15/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1313 - accuracy: 0.9452 - val_loss: 0.1076 - val_accuracy: 0.9638 Epoch 16/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1231 - accuracy: 0.9420 - val_loss: 0.1036 - val_accuracy: 0.9638 Epoch 17/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1339 - accuracy: 0.9396 - val_loss: 0.0997 - val_accuracy: 0.9710 Epoch 18/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1269 - accuracy: 0.9493 - val_loss: 0.0935 - val_accuracy: 0.9710 Epoch 19/135 10/10 [==============================] - 0s 13ms/step - loss: 0.1339 - accuracy: 0.9493 - val_loss: 0.0909 - val_accuracy: 0.9710 Epoch 20/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1276 - accuracy: 0.9477 - val_loss: 0.0880 - val_accuracy: 0.9710 Epoch 21/135 10/10 [==============================] - 0s 9ms/step - loss: 0.1207 - accuracy: 0.9525 - val_loss: 0.0843 - val_accuracy: 0.9710 Epoch 22/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1221 - accuracy: 0.9501 - val_loss: 0.0809 - val_accuracy: 0.9710 Epoch 23/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1062 - accuracy: 0.9541 - val_loss: 0.0788 - val_accuracy: 0.9710 Epoch 24/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1019 - accuracy: 0.9597 - val_loss: 0.0775 - val_accuracy: 0.9710 Epoch 25/135 10/10 [==============================] - 0s 9ms/step - loss: 0.1038 - accuracy: 0.9605 - val_loss: 0.0768 - val_accuracy: 0.9638 Epoch 26/135 10/10 [==============================] - 0s 11ms/step - loss: 0.1023 - accuracy: 0.9654 - val_loss: 0.0761 - val_accuracy: 0.9710 Epoch 27/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0942 - accuracy: 0.9605 - val_loss: 0.0743 - val_accuracy: 0.9710 Epoch 28/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0974 - accuracy: 0.9614 - val_loss: 0.0712 - val_accuracy: 0.9710 Epoch 29/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0885 - accuracy: 0.9589 - val_loss: 0.0690 - val_accuracy: 0.9710 Epoch 30/135 10/10 [==============================] - 0s 14ms/step - loss: 0.0877 - accuracy: 0.9646 - val_loss: 0.0661 - val_accuracy: 0.9710 Epoch 31/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0958 - accuracy: 0.9662 - val_loss: 0.0651 - val_accuracy: 0.9710 Epoch 32/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0840 - accuracy: 0.9630 - val_loss: 0.0636 - val_accuracy: 0.9783 Epoch 33/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0885 - accuracy: 0.9630 - val_loss: 0.0635 - val_accuracy: 0.9783 Epoch 34/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0906 - accuracy: 0.9630 - val_loss: 0.0618 - val_accuracy: 0.9783 Epoch 35/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0896 - accuracy: 0.9622 - val_loss: 0.0615 - val_accuracy: 0.9928 Epoch 36/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0716 - accuracy: 0.9758 - val_loss: 0.0602 - val_accuracy: 0.9928 Epoch 37/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0799 - accuracy: 0.9662 - val_loss: 0.0593 - val_accuracy: 0.9855 Epoch 38/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0718 - accuracy: 0.9710 - val_loss: 0.0593 - val_accuracy: 0.9928 Epoch 39/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0777 - accuracy: 0.9710 - val_loss: 0.0591 - val_accuracy: 0.9855 Epoch 40/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0735 - accuracy: 0.9742 - val_loss: 0.0617 - val_accuracy: 0.9855 Epoch 41/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0692 - accuracy: 0.9710 - val_loss: 0.0620 - val_accuracy: 0.9855 Epoch 42/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0761 - accuracy: 0.9767 - val_loss: 0.0603 - val_accuracy: 0.9783 Epoch 43/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0614 - accuracy: 0.9775 - val_loss: 0.0590 - val_accuracy: 0.9855 Epoch 44/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0714 - accuracy: 0.9710 - val_loss: 0.0587 - val_accuracy: 0.9783 Epoch 45/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0525 - accuracy: 0.9839 - val_loss: 0.0613 - val_accuracy: 0.9783 Epoch 46/135 10/10 [==============================] - 0s 15ms/step - loss: 0.0662 - accuracy: 0.9799 - val_loss: 0.0617 - val_accuracy: 0.9783 Epoch 47/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0537 - accuracy: 0.9783 - val_loss: 0.0610 - val_accuracy: 0.9855 Epoch 48/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0680 - accuracy: 0.9734 - val_loss: 0.0613 - val_accuracy: 0.9855 Epoch 49/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0670 - accuracy: 0.9750 - val_loss: 0.0612 - val_accuracy: 0.9783 Epoch 50/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0628 - accuracy: 0.9742 - val_loss: 0.0599 - val_accuracy: 0.9855 Epoch 51/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0540 - accuracy: 0.9823 - val_loss: 0.0607 - val_accuracy: 0.9855 Epoch 52/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0552 - accuracy: 0.9807 - val_loss: 0.0633 - val_accuracy: 0.9855 Epoch 53/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0483 - accuracy: 0.9855 - val_loss: 0.0629 - val_accuracy: 0.9783 Epoch 54/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0465 - accuracy: 0.9815 - val_loss: 0.0615 - val_accuracy: 0.9783 Epoch 55/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0483 - accuracy: 0.9839 - val_loss: 0.0628 - val_accuracy: 0.9783 Epoch 56/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0571 - accuracy: 0.9783 - val_loss: 0.0618 - val_accuracy: 0.9855 Epoch 57/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0642 - accuracy: 0.9807 - val_loss: 0.0586 - val_accuracy: 0.9783 Epoch 58/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0490 - accuracy: 0.9823 - val_loss: 0.0588 - val_accuracy: 0.9783 Epoch 59/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0436 - accuracy: 0.9855 - val_loss: 0.0608 - val_accuracy: 0.9783 Epoch 60/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0427 - accuracy: 0.9879 - val_loss: 0.0627 - val_accuracy: 0.9783 Epoch 61/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0331 - accuracy: 0.9911 - val_loss: 0.0615 - val_accuracy: 0.9783 Epoch 62/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0402 - accuracy: 0.9847 - val_loss: 0.0603 - val_accuracy: 0.9855 Epoch 63/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0431 - accuracy: 0.9807 - val_loss: 0.0603 - val_accuracy: 0.9783 Epoch 64/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0384 - accuracy: 0.9879 - val_loss: 0.0593 - val_accuracy: 0.9783 Epoch 65/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0320 - accuracy: 0.9936 - val_loss: 0.0627 - val_accuracy: 0.9783 Epoch 66/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0381 - accuracy: 0.9871 - val_loss: 0.0641 - val_accuracy: 0.9783 Epoch 67/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0482 - accuracy: 0.9839 - val_loss: 0.0642 - val_accuracy: 0.9783 Epoch 68/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0447 - accuracy: 0.9823 - val_loss: 0.0725 - val_accuracy: 0.9710 Epoch 69/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0434 - accuracy: 0.9871 - val_loss: 0.0763 - val_accuracy: 0.9710 Epoch 70/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0351 - accuracy: 0.9887 - val_loss: 0.0724 - val_accuracy: 0.9783 Epoch 71/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0306 - accuracy: 0.9895 - val_loss: 0.0739 - val_accuracy: 0.9783 Epoch 72/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0392 - accuracy: 0.9823 - val_loss: 0.0757 - val_accuracy: 0.9710 Epoch 73/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0528 - accuracy: 0.9831 - val_loss: 0.0739 - val_accuracy: 0.9710 Epoch 74/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0306 - accuracy: 0.9871 - val_loss: 0.0695 - val_accuracy: 0.9783 Epoch 75/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0310 - accuracy: 0.9903 - val_loss: 0.0681 - val_accuracy: 0.9783 Epoch 76/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0275 - accuracy: 0.9895 - val_loss: 0.0638 - val_accuracy: 0.9783 Epoch 77/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0357 - accuracy: 0.9879 - val_loss: 0.0591 - val_accuracy: 0.9855 Epoch 78/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0436 - accuracy: 0.9839 - val_loss: 0.0600 - val_accuracy: 0.9855 Epoch 79/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0373 - accuracy: 0.9879 - val_loss: 0.0624 - val_accuracy: 0.9855 Epoch 80/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0331 - accuracy: 0.9911 - val_loss: 0.0621 - val_accuracy: 0.9783 Epoch 81/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0316 - accuracy: 0.9879 - val_loss: 0.0646 - val_accuracy: 0.9783 Epoch 82/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0318 - accuracy: 0.9887 - val_loss: 0.0679 - val_accuracy: 0.9783 Epoch 83/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0244 - accuracy: 0.9919 - val_loss: 0.0701 - val_accuracy: 0.9783 Epoch 84/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0275 - accuracy: 0.9911 - val_loss: 0.0761 - val_accuracy: 0.9783 Epoch 85/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0206 - accuracy: 0.9928 - val_loss: 0.0856 - val_accuracy: 0.9710 Epoch 86/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0295 - accuracy: 0.9839 - val_loss: 0.0837 - val_accuracy: 0.9710 Epoch 87/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0205 - accuracy: 0.9936 - val_loss: 0.0885 - val_accuracy: 0.9710 Epoch 88/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0233 - accuracy: 0.9928 - val_loss: 0.0914 - val_accuracy: 0.9710 Epoch 89/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0219 - accuracy: 0.9919 - val_loss: 0.0975 - val_accuracy: 0.9710 Epoch 90/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0254 - accuracy: 0.9911 - val_loss: 0.1021 - val_accuracy: 0.9638 Epoch 91/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0269 - accuracy: 0.9879 - val_loss: 0.1029 - val_accuracy: 0.9710 Epoch 92/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0288 - accuracy: 0.9887 - val_loss: 0.0960 - val_accuracy: 0.9710 Epoch 93/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0219 - accuracy: 0.9952 - val_loss: 0.0955 - val_accuracy: 0.9710 Epoch 94/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0235 - accuracy: 0.9911 - val_loss: 0.0984 - val_accuracy: 0.9638 Epoch 95/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0263 - accuracy: 0.9871 - val_loss: 0.1065 - val_accuracy: 0.9638 Epoch 96/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0209 - accuracy: 0.9936 - val_loss: 0.1132 - val_accuracy: 0.9638 Epoch 97/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0245 - accuracy: 0.9911 - val_loss: 0.1158 - val_accuracy: 0.9638 Epoch 98/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0222 - accuracy: 0.9911 - val_loss: 0.1084 - val_accuracy: 0.9710 Epoch 99/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0216 - accuracy: 0.9928 - val_loss: 0.1047 - val_accuracy: 0.9710 Epoch 100/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0194 - accuracy: 0.9919 - val_loss: 0.0963 - val_accuracy: 0.9783 Epoch 101/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0270 - accuracy: 0.9895 - val_loss: 0.1049 - val_accuracy: 0.9710 Epoch 102/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0246 - accuracy: 0.9928 - val_loss: 0.1129 - val_accuracy: 0.9710 Epoch 103/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0207 - accuracy: 0.9903 - val_loss: 0.1136 - val_accuracy: 0.9710 Epoch 104/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0228 - accuracy: 0.9928 - val_loss: 0.1098 - val_accuracy: 0.9710 Epoch 105/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0166 - accuracy: 0.9960 - val_loss: 0.1091 - val_accuracy: 0.9710 Epoch 106/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0199 - accuracy: 0.9944 - val_loss: 0.1028 - val_accuracy: 0.9783 Epoch 107/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0207 - accuracy: 0.9911 - val_loss: 0.1015 - val_accuracy: 0.9783 Epoch 108/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0146 - accuracy: 0.9944 - val_loss: 0.1066 - val_accuracy: 0.9710 Epoch 109/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0179 - accuracy: 0.9952 - val_loss: 0.1069 - val_accuracy: 0.9783 Epoch 110/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0232 - accuracy: 0.9928 - val_loss: 0.1129 - val_accuracy: 0.9783 Epoch 111/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0090 - accuracy: 0.9968 - val_loss: 0.1138 - val_accuracy: 0.9710 Epoch 112/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0153 - accuracy: 0.9960 - val_loss: 0.1127 - val_accuracy: 0.9710 Epoch 113/135 10/10 [==============================] - 0s 7ms/step - loss: 0.0129 - accuracy: 0.9960 - val_loss: 0.1142 - val_accuracy: 0.9710 Epoch 114/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0221 - accuracy: 0.9952 - val_loss: 0.1053 - val_accuracy: 0.9783 Epoch 115/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0152 - accuracy: 0.9952 - val_loss: 0.1040 - val_accuracy: 0.9783 Epoch 116/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0140 - accuracy: 0.9968 - val_loss: 0.1076 - val_accuracy: 0.9783 Epoch 117/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0190 - accuracy: 0.9936 - val_loss: 0.1167 - val_accuracy: 0.9710 Epoch 118/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0262 - accuracy: 0.9936 - val_loss: 0.1108 - val_accuracy: 0.9783 Epoch 119/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0137 - accuracy: 0.9944 - val_loss: 0.1110 - val_accuracy: 0.9710 Epoch 120/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0120 - accuracy: 0.9960 - val_loss: 0.1112 - val_accuracy: 0.9710 Epoch 121/135 10/10 [==============================] - 0s 8ms/step - loss: 0.0090 - accuracy: 0.9976 - val_loss: 0.1107 - val_accuracy: 0.9710 Epoch 122/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0133 - accuracy: 0.9976 - val_loss: 0.1157 - val_accuracy: 0.9710 Epoch 123/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0092 - accuracy: 0.9968 - val_loss: 0.1127 - val_accuracy: 0.9783 Epoch 124/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0148 - accuracy: 0.9944 - val_loss: 0.1174 - val_accuracy: 0.9783 Epoch 125/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0154 - accuracy: 0.9960 - val_loss: 0.1173 - val_accuracy: 0.9783 Epoch 126/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0139 - accuracy: 0.9952 - val_loss: 0.1188 - val_accuracy: 0.9783 Epoch 127/135 10/10 [==============================] - 0s 13ms/step - loss: 0.0115 - accuracy: 0.9968 - val_loss: 0.1287 - val_accuracy: 0.9710 Epoch 128/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0095 - accuracy: 0.9968 - val_loss: 0.1337 - val_accuracy: 0.9710 Epoch 129/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0100 - accuracy: 0.9968 - val_loss: 0.1370 - val_accuracy: 0.9710 Epoch 130/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0132 - accuracy: 0.9976 - val_loss: 0.1362 - val_accuracy: 0.9710 Epoch 131/135 10/10 [==============================] - 0s 10ms/step - loss: 0.0210 - accuracy: 0.9936 - val_loss: 0.1297 - val_accuracy: 0.9638 Epoch 132/135 10/10 [==============================] - 0s 12ms/step - loss: 0.0087 - accuracy: 0.9976 - val_loss: 0.1078 - val_accuracy: 0.9783 Epoch 133/135 10/10 [==============================] - 0s 11ms/step - loss: 0.0117 - accuracy: 0.9960 - val_loss: 0.1106 - val_accuracy: 0.9783 Epoch 134/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0096 - accuracy: 0.9984 - val_loss: 0.1140 - val_accuracy: 0.9783 Epoch 135/135 10/10 [==============================] - 0s 9ms/step - loss: 0.0090 - accuracy: 0.9976 - val_loss: 0.1187 - val_accuracy: 0.9783 Training completed in time: 0:00:14.978126
score = model.evaluate(X_train_mix, y_train_mix, verbose=0)
print("Training Accuracy: ", score[1])
score = model.evaluate(X_test_mix, y_test_mix, verbose=0)
print("Testing Accuracy: ", score[1])
Training Accuracy: 0.997826099395752 Testing Accuracy: 0.796875
# plot training history
from matplotlib import pyplot
pyplot.plot(history.history['loss'], label='train')
pyplot.plot(history.history['val_loss'], label='val')
pyplot.legend()
pyplot.show()
plt.plot(history.history['accuracy'], label='train acc')
plt.plot(history.history['val_accuracy'], label='val acc')
plt.legend()
plt.show()
plt.savefig('AccVal_acc')
<Figure size 432x288 with 0 Axes>
print("Classification Report for Test Data\n")
report(X_test_mix, y_test_mix)
Classification Report for Test Data
Classification Report
precision recall f1-score support
0 0.88 0.73 0.80 105
1 0.73 0.87 0.80 87
accuracy 0.80 192
macro avg 0.80 0.80 0.80 192
weighted avg 0.81 0.80 0.80 192
def create_cnn_model(input_shape):
model = models.Sequential()
# Convolutional layers
model.add(layers.Conv1D(64, 5, activation='relu', input_shape=(input_shape, 1)))
model.add(layers.MaxPooling1D(3))
model.add(layers.Conv1D(64, 5, activation='relu'))
model.add(layers.MaxPooling1D(3))
# Flattening the outputs for the dense layers
model.add(layers.Flatten())
# Dense layers
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dropout(0.5))
# Output layer
model.add(layers.Dense(2, activation='softmax'))
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
return model
from tensorflow.keras.utils import to_categorical
y_train_mix = to_categorical(y_train_mix, num_classes=2)
y_test_mix = to_categorical(y_test_mix, num_classes=2)
input_shape = X_train_mix.shape[1]
cnn_model = create_cnn_model(input_shape)
cnn_model.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d (Conv1D) (None, 21, 64) 384
max_pooling1d (MaxPooling1D (None, 7, 64) 0
)
conv1d_1 (Conv1D) (None, 3, 64) 20544
max_pooling1d_1 (MaxPooling (None, 1, 64) 0
1D)
flatten (Flatten) (None, 64) 0
dense_8 (Dense) (None, 256) 16640
dropout_8 (Dropout) (None, 256) 0
dense_9 (Dense) (None, 128) 32896
dropout_9 (Dropout) (None, 128) 0
dense_10 (Dense) (None, 64) 8256
dropout_10 (Dropout) (None, 64) 0
dense_11 (Dense) (None, 2) 130
=================================================================
Total params: 78,850
Trainable params: 78,850
Non-trainable params: 0
_________________________________________________________________
# Calculate pre-training accuracy
score = cnn_model.evaluate(X_test_mix, y_test_mix, verbose=1)
accuracy = 100*score[1]
print("Pre-training accuracy: %.4f%%" % accuracy)
6/6 [==============================] - 0s 3ms/step - loss: 0.6994 - accuracy: 0.3906 Pre-training accuracy: 39.0625%
start = datetime.now()
#training
history = cnn_model.fit(X_train_mix,
y_train_mix,
validation_split=(0.1),
epochs=60,
batch_size=128)
duration = datetime.now() - start
print("Training completed in time: ", duration)
Epoch 1/60 10/10 [==============================] - 1s 38ms/step - loss: 0.6815 - accuracy: 0.5515 - val_loss: 0.6301 - val_accuracy: 0.7246 Epoch 2/60 10/10 [==============================] - 0s 12ms/step - loss: 0.6023 - accuracy: 0.6892 - val_loss: 0.5256 - val_accuracy: 0.7609 Epoch 3/60 10/10 [==============================] - 0s 13ms/step - loss: 0.5293 - accuracy: 0.7528 - val_loss: 0.4382 - val_accuracy: 0.8043 Epoch 4/60 10/10 [==============================] - 0s 12ms/step - loss: 0.4509 - accuracy: 0.8060 - val_loss: 0.3716 - val_accuracy: 0.8551 Epoch 5/60 10/10 [==============================] - 0s 13ms/step - loss: 0.3913 - accuracy: 0.8333 - val_loss: 0.3019 - val_accuracy: 0.8913 Epoch 6/60 10/10 [==============================] - 0s 13ms/step - loss: 0.3031 - accuracy: 0.8889 - val_loss: 0.2464 - val_accuracy: 0.9058 Epoch 7/60 10/10 [==============================] - 0s 13ms/step - loss: 0.2509 - accuracy: 0.9066 - val_loss: 0.2245 - val_accuracy: 0.8913 Epoch 8/60 10/10 [==============================] - 0s 16ms/step - loss: 0.2276 - accuracy: 0.9138 - val_loss: 0.2395 - val_accuracy: 0.9058 Epoch 9/60 10/10 [==============================] - 0s 15ms/step - loss: 0.1997 - accuracy: 0.9340 - val_loss: 0.1684 - val_accuracy: 0.9203 Epoch 10/60 10/10 [==============================] - 0s 19ms/step - loss: 0.1662 - accuracy: 0.9452 - val_loss: 0.1181 - val_accuracy: 0.9493 Epoch 11/60 10/10 [==============================] - 0s 14ms/step - loss: 0.1329 - accuracy: 0.9589 - val_loss: 0.0978 - val_accuracy: 0.9638 Epoch 12/60 10/10 [==============================] - 0s 13ms/step - loss: 0.1124 - accuracy: 0.9646 - val_loss: 0.1776 - val_accuracy: 0.9493 Epoch 13/60 10/10 [==============================] - 0s 13ms/step - loss: 0.1247 - accuracy: 0.9573 - val_loss: 0.1224 - val_accuracy: 0.9565 Epoch 14/60 10/10 [==============================] - 0s 12ms/step - loss: 0.1021 - accuracy: 0.9678 - val_loss: 0.0799 - val_accuracy: 0.9783 Epoch 15/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0898 - accuracy: 0.9718 - val_loss: 0.1056 - val_accuracy: 0.9638 Epoch 16/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0870 - accuracy: 0.9686 - val_loss: 0.1031 - val_accuracy: 0.9493 Epoch 17/60 10/10 [==============================] - 0s 12ms/step - loss: 0.0749 - accuracy: 0.9783 - val_loss: 0.0853 - val_accuracy: 0.9710 Epoch 18/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0784 - accuracy: 0.9758 - val_loss: 0.0664 - val_accuracy: 0.9783 Epoch 19/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0508 - accuracy: 0.9911 - val_loss: 0.0936 - val_accuracy: 0.9710 Epoch 20/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0523 - accuracy: 0.9839 - val_loss: 0.0927 - val_accuracy: 0.9638 Epoch 21/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0545 - accuracy: 0.9839 - val_loss: 0.1030 - val_accuracy: 0.9493 Epoch 22/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0380 - accuracy: 0.9919 - val_loss: 0.1028 - val_accuracy: 0.9710 Epoch 23/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0389 - accuracy: 0.9919 - val_loss: 0.1023 - val_accuracy: 0.9783 Epoch 24/60 10/10 [==============================] - 0s 19ms/step - loss: 0.0244 - accuracy: 0.9960 - val_loss: 0.0861 - val_accuracy: 0.9710 Epoch 25/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0205 - accuracy: 0.9960 - val_loss: 0.1106 - val_accuracy: 0.9783 Epoch 26/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0213 - accuracy: 0.9952 - val_loss: 0.1223 - val_accuracy: 0.9638 Epoch 27/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0276 - accuracy: 0.9952 - val_loss: 0.1414 - val_accuracy: 0.9638 Epoch 28/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0202 - accuracy: 0.9944 - val_loss: 0.0864 - val_accuracy: 0.9783 Epoch 29/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0264 - accuracy: 0.9919 - val_loss: 0.1277 - val_accuracy: 0.9493 Epoch 30/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0348 - accuracy: 0.9879 - val_loss: 0.1028 - val_accuracy: 0.9710 Epoch 31/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0186 - accuracy: 0.9968 - val_loss: 0.1176 - val_accuracy: 0.9638 Epoch 32/60 10/10 [==============================] - 0s 18ms/step - loss: 0.0123 - accuracy: 0.9960 - val_loss: 0.1629 - val_accuracy: 0.9710 Epoch 33/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0268 - accuracy: 0.9895 - val_loss: 0.1999 - val_accuracy: 0.9710 Epoch 34/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0161 - accuracy: 0.9968 - val_loss: 0.1564 - val_accuracy: 0.9638 Epoch 35/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0142 - accuracy: 0.9984 - val_loss: 0.1657 - val_accuracy: 0.9710 Epoch 36/60 10/10 [==============================] - 0s 19ms/step - loss: 0.0249 - accuracy: 0.9928 - val_loss: 0.1769 - val_accuracy: 0.9638 Epoch 37/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0171 - accuracy: 0.9960 - val_loss: 0.1329 - val_accuracy: 0.9638 Epoch 38/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0365 - accuracy: 0.9903 - val_loss: 0.1534 - val_accuracy: 0.9638 Epoch 39/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0216 - accuracy: 0.9944 - val_loss: 0.1022 - val_accuracy: 0.9855 Epoch 40/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0179 - accuracy: 0.9919 - val_loss: 0.1141 - val_accuracy: 0.9783 Epoch 41/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0173 - accuracy: 0.9936 - val_loss: 0.0754 - val_accuracy: 0.9638 Epoch 42/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0109 - accuracy: 0.9976 - val_loss: 0.1589 - val_accuracy: 0.9638 Epoch 43/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0088 - accuracy: 0.9976 - val_loss: 0.1025 - val_accuracy: 0.9783 Epoch 44/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0062 - accuracy: 0.9984 - val_loss: 0.1003 - val_accuracy: 0.9710 Epoch 45/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0105 - accuracy: 0.9960 - val_loss: 0.1944 - val_accuracy: 0.9710 Epoch 46/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0272 - accuracy: 0.9936 - val_loss: 0.1183 - val_accuracy: 0.9710 Epoch 47/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0184 - accuracy: 0.9952 - val_loss: 0.1247 - val_accuracy: 0.9710 Epoch 48/60 10/10 [==============================] - 0s 16ms/step - loss: 0.0228 - accuracy: 0.9928 - val_loss: 0.1067 - val_accuracy: 0.9638 Epoch 49/60 10/10 [==============================] - 0s 18ms/step - loss: 0.0121 - accuracy: 0.9968 - val_loss: 0.1760 - val_accuracy: 0.9710 Epoch 50/60 10/10 [==============================] - 0s 19ms/step - loss: 0.0089 - accuracy: 0.9976 - val_loss: 0.0598 - val_accuracy: 0.9783 Epoch 51/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0054 - accuracy: 0.9992 - val_loss: 0.0679 - val_accuracy: 0.9855 Epoch 52/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0026 - accuracy: 0.9992 - val_loss: 0.0568 - val_accuracy: 0.9928 Epoch 53/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0015 - accuracy: 1.0000 - val_loss: 0.0998 - val_accuracy: 0.9710 Epoch 54/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.0858 - val_accuracy: 0.9710 Epoch 55/60 10/10 [==============================] - 0s 15ms/step - loss: 0.0026 - accuracy: 0.9992 - val_loss: 0.1737 - val_accuracy: 0.9710 Epoch 56/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0037 - accuracy: 0.9992 - val_loss: 0.0884 - val_accuracy: 0.9783 Epoch 57/60 10/10 [==============================] - 0s 13ms/step - loss: 0.0014 - accuracy: 1.0000 - val_loss: 0.2088 - val_accuracy: 0.9710 Epoch 58/60 10/10 [==============================] - 0s 14ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.1425 - val_accuracy: 0.9710 Epoch 59/60 10/10 [==============================] - 0s 17ms/step - loss: 0.0021 - accuracy: 0.9992 - val_loss: 0.2911 - val_accuracy: 0.9710 Epoch 60/60 10/10 [==============================] - 0s 22ms/step - loss: 0.0022 - accuracy: 0.9984 - val_loss: 0.1333 - val_accuracy: 0.9710 Training completed in time: 0:00:10.208697
score = cnn_model.evaluate(X_train_mix, y_train_mix, verbose=0)
print("Training Accuracy: ", score[1])
score = cnn_model.evaluate(X_test_mix, y_test_mix, verbose=0)
print("Testing Accuracy: ", score[1])
Training Accuracy: 0.9963768124580383 Testing Accuracy: 0.8229166865348816
Increased Dropout Rate: The dropout rate has been increased to 0.6 after each convolutional and dense layer.
L2 Regularization: Added L2 regularization to convolutional and dense layers, which penalizes large weights.
Early Stopping: An early stopping callback is set to monitor validation loss and will stop training if the loss does not decrease for 5 consecutive epochs.
from tensorflow.keras import models, layers, regularizers
from tensorflow.keras.callbacks import EarlyStopping
def create_improved_cnn_model(input_shape):
model = models.Sequential()
# Convolutional layers with increased dropout and L2 regularization
model.add(layers.Conv1D(64, 5, activation='relu', input_shape=(input_shape, 1),
kernel_regularizer=regularizers.l2(0.001)))
model.add(layers.MaxPooling1D(3))
model.add(layers.Dropout(0.6))
model.add(layers.Conv1D(64, 5, activation='relu',
kernel_regularizer=regularizers.l2(0.001)))
model.add(layers.MaxPooling1D(3))
model.add(layers.Dropout(0.6))
# Flattening the outputs for the dense layers
model.add(layers.Flatten())
# Dense layers with increased dropout
model.add(layers.Dense(256, activation='relu', kernel_regularizer=regularizers.l2(0.001)))
model.add(layers.Dropout(0.6))
model.add(layers.Dense(128, activation='relu', kernel_regularizer=regularizers.l2(0.001)))
model.add(layers.Dropout(0.6))
model.add(layers.Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.001)))
model.add(layers.Dropout(0.6))
# Output layer
model.add(layers.Dense(2, activation='softmax'))
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
return model
# Early Stopping Criterion
early_stopping = EarlyStopping(monitor='val_loss', patience=5)
input_shape = X_train_mix.shape[1]
imp_cnn_model = create_improved_cnn_model(input_shape)
imp_cnn_model.summary()
# Calculate pre-training accuracy
score = imp_cnn_model.evaluate(X_test_mix, y_test_mix, verbose=1)
accuracy = 100*score[1]
print("Pre-training accuracy: %.4f%%" % accuracy)
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_2 (Conv1D) (None, 21, 64) 384
max_pooling1d_2 (MaxPooling (None, 7, 64) 0
1D)
dropout_11 (Dropout) (None, 7, 64) 0
conv1d_3 (Conv1D) (None, 3, 64) 20544
max_pooling1d_3 (MaxPooling (None, 1, 64) 0
1D)
dropout_12 (Dropout) (None, 1, 64) 0
flatten_1 (Flatten) (None, 64) 0
dense_12 (Dense) (None, 256) 16640
dropout_13 (Dropout) (None, 256) 0
dense_13 (Dense) (None, 128) 32896
dropout_14 (Dropout) (None, 128) 0
dense_14 (Dense) (None, 64) 8256
dropout_15 (Dropout) (None, 64) 0
dense_15 (Dense) (None, 2) 130
=================================================================
Total params: 78,850
Trainable params: 78,850
Non-trainable params: 0
_________________________________________________________________
6/6 [==============================] - 0s 3ms/step - loss: 1.1089 - accuracy: 0.5469
Pre-training accuracy: 54.6875%
# To train the model, use:
imp_cnn_model.fit(X_train_mix, y_train_mix, epochs=60, validation_split=0.2, callbacks=[early_stopping])
Epoch 1/60 35/35 [==============================] - 1s 17ms/step - loss: 1.1493 - accuracy: 0.4891 - val_loss: 1.0939 - val_accuracy: 0.4710 Epoch 2/60 35/35 [==============================] - 0s 8ms/step - loss: 1.0878 - accuracy: 0.5199 - val_loss: 1.0703 - val_accuracy: 0.4855 Epoch 3/60 35/35 [==============================] - 0s 8ms/step - loss: 1.0606 - accuracy: 0.5489 - val_loss: 1.0458 - val_accuracy: 0.6051 Epoch 4/60 35/35 [==============================] - 0s 8ms/step - loss: 1.0244 - accuracy: 0.5507 - val_loss: 0.9960 - val_accuracy: 0.7319 Epoch 5/60 35/35 [==============================] - 0s 8ms/step - loss: 0.9679 - accuracy: 0.6486 - val_loss: 0.9247 - val_accuracy: 0.7283 Epoch 6/60 35/35 [==============================] - 0s 8ms/step - loss: 0.9147 - accuracy: 0.7101 - val_loss: 0.8823 - val_accuracy: 0.7065 Epoch 7/60 35/35 [==============================] - 0s 8ms/step - loss: 0.8746 - accuracy: 0.7156 - val_loss: 0.8691 - val_accuracy: 0.6920 Epoch 8/60 35/35 [==============================] - 0s 7ms/step - loss: 0.8404 - accuracy: 0.7355 - val_loss: 0.7965 - val_accuracy: 0.7319 Epoch 9/60 35/35 [==============================] - 0s 7ms/step - loss: 0.8257 - accuracy: 0.7500 - val_loss: 0.8261 - val_accuracy: 0.6957 Epoch 10/60 35/35 [==============================] - 0s 9ms/step - loss: 0.7924 - accuracy: 0.7591 - val_loss: 0.7540 - val_accuracy: 0.7754 Epoch 11/60 35/35 [==============================] - 0s 8ms/step - loss: 0.7697 - accuracy: 0.7609 - val_loss: 0.7173 - val_accuracy: 0.7862 Epoch 12/60 35/35 [==============================] - 0s 9ms/step - loss: 0.7411 - accuracy: 0.7699 - val_loss: 0.6733 - val_accuracy: 0.8043 Epoch 13/60 35/35 [==============================] - 0s 9ms/step - loss: 0.7005 - accuracy: 0.7844 - val_loss: 0.6394 - val_accuracy: 0.8370 Epoch 14/60 35/35 [==============================] - 0s 10ms/step - loss: 0.6908 - accuracy: 0.7989 - val_loss: 0.5890 - val_accuracy: 0.8587 Epoch 15/60 35/35 [==============================] - 0s 9ms/step - loss: 0.6551 - accuracy: 0.8216 - val_loss: 0.5595 - val_accuracy: 0.8587 Epoch 16/60 35/35 [==============================] - 0s 9ms/step - loss: 0.6316 - accuracy: 0.8216 - val_loss: 0.5548 - val_accuracy: 0.8659 Epoch 17/60 35/35 [==============================] - 0s 8ms/step - loss: 0.6127 - accuracy: 0.8297 - val_loss: 0.5237 - val_accuracy: 0.8696 Epoch 18/60 35/35 [==============================] - 0s 8ms/step - loss: 0.6073 - accuracy: 0.8270 - val_loss: 0.5296 - val_accuracy: 0.8623 Epoch 19/60 35/35 [==============================] - 0s 9ms/step - loss: 0.5888 - accuracy: 0.8505 - val_loss: 0.4983 - val_accuracy: 0.8841 Epoch 20/60 35/35 [==============================] - 0s 9ms/step - loss: 0.5771 - accuracy: 0.8324 - val_loss: 0.4938 - val_accuracy: 0.8804 Epoch 21/60 35/35 [==============================] - 0s 7ms/step - loss: 0.5568 - accuracy: 0.8469 - val_loss: 0.4947 - val_accuracy: 0.8732 Epoch 22/60 35/35 [==============================] - 0s 7ms/step - loss: 0.5292 - accuracy: 0.8505 - val_loss: 0.4370 - val_accuracy: 0.8949 Epoch 23/60 35/35 [==============================] - 0s 7ms/step - loss: 0.5430 - accuracy: 0.8478 - val_loss: 0.4590 - val_accuracy: 0.8913 Epoch 24/60 35/35 [==============================] - 0s 8ms/step - loss: 0.5192 - accuracy: 0.8587 - val_loss: 0.4284 - val_accuracy: 0.9022 Epoch 25/60 35/35 [==============================] - 0s 8ms/step - loss: 0.5087 - accuracy: 0.8406 - val_loss: 0.4236 - val_accuracy: 0.8986 Epoch 26/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4980 - accuracy: 0.8596 - val_loss: 0.4286 - val_accuracy: 0.8768 Epoch 27/60 35/35 [==============================] - 0s 9ms/step - loss: 0.4805 - accuracy: 0.8678 - val_loss: 0.4116 - val_accuracy: 0.8949 Epoch 28/60 35/35 [==============================] - 0s 9ms/step - loss: 0.4868 - accuracy: 0.8460 - val_loss: 0.4021 - val_accuracy: 0.9167 Epoch 29/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4655 - accuracy: 0.8650 - val_loss: 0.3785 - val_accuracy: 0.8986 Epoch 30/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4238 - accuracy: 0.8895 - val_loss: 0.3879 - val_accuracy: 0.8913 Epoch 31/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4574 - accuracy: 0.8687 - val_loss: 0.3661 - val_accuracy: 0.9094 Epoch 32/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4502 - accuracy: 0.8705 - val_loss: 0.3577 - val_accuracy: 0.9094 Epoch 33/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4157 - accuracy: 0.8877 - val_loss: 0.3384 - val_accuracy: 0.9239 Epoch 34/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4229 - accuracy: 0.8913 - val_loss: 0.3680 - val_accuracy: 0.8949 Epoch 35/60 35/35 [==============================] - 0s 8ms/step - loss: 0.4074 - accuracy: 0.8967 - val_loss: 0.3433 - val_accuracy: 0.9058 Epoch 36/60 35/35 [==============================] - 0s 7ms/step - loss: 0.4165 - accuracy: 0.8777 - val_loss: 0.3313 - val_accuracy: 0.9130 Epoch 37/60 35/35 [==============================] - 0s 7ms/step - loss: 0.4031 - accuracy: 0.8859 - val_loss: 0.3454 - val_accuracy: 0.9022 Epoch 38/60 35/35 [==============================] - 0s 7ms/step - loss: 0.3950 - accuracy: 0.8859 - val_loss: 0.3336 - val_accuracy: 0.9130 Epoch 39/60 35/35 [==============================] - 0s 7ms/step - loss: 0.3698 - accuracy: 0.9067 - val_loss: 0.3262 - val_accuracy: 0.9094 Epoch 40/60 35/35 [==============================] - 0s 7ms/step - loss: 0.4183 - accuracy: 0.8859 - val_loss: 0.3376 - val_accuracy: 0.9167 Epoch 41/60 35/35 [==============================] - 0s 10ms/step - loss: 0.3896 - accuracy: 0.8904 - val_loss: 0.3150 - val_accuracy: 0.9275 Epoch 42/60 35/35 [==============================] - 0s 10ms/step - loss: 0.3799 - accuracy: 0.8976 - val_loss: 0.2979 - val_accuracy: 0.9239 Epoch 43/60 35/35 [==============================] - 0s 10ms/step - loss: 0.3638 - accuracy: 0.8931 - val_loss: 0.3092 - val_accuracy: 0.9130 Epoch 44/60 35/35 [==============================] - 0s 10ms/step - loss: 0.3807 - accuracy: 0.8904 - val_loss: 0.3209 - val_accuracy: 0.9130 Epoch 45/60 35/35 [==============================] - 0s 8ms/step - loss: 0.3854 - accuracy: 0.8886 - val_loss: 0.3220 - val_accuracy: 0.9022 Epoch 46/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3737 - accuracy: 0.8913 - val_loss: 0.3057 - val_accuracy: 0.9275 Epoch 47/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3685 - accuracy: 0.8868 - val_loss: 0.2976 - val_accuracy: 0.9239 Epoch 48/60 35/35 [==============================] - 0s 8ms/step - loss: 0.3676 - accuracy: 0.8895 - val_loss: 0.3015 - val_accuracy: 0.9203 Epoch 49/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3520 - accuracy: 0.8967 - val_loss: 0.2843 - val_accuracy: 0.9312 Epoch 50/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3480 - accuracy: 0.9004 - val_loss: 0.2870 - val_accuracy: 0.9239 Epoch 51/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3543 - accuracy: 0.9013 - val_loss: 0.3061 - val_accuracy: 0.8986 Epoch 52/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3527 - accuracy: 0.9004 - val_loss: 0.2946 - val_accuracy: 0.9203 Epoch 53/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3645 - accuracy: 0.8913 - val_loss: 0.2788 - val_accuracy: 0.9239 Epoch 54/60 35/35 [==============================] - 0s 10ms/step - loss: 0.3535 - accuracy: 0.8958 - val_loss: 0.2880 - val_accuracy: 0.9167 Epoch 55/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3372 - accuracy: 0.9022 - val_loss: 0.2778 - val_accuracy: 0.9384 Epoch 56/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3398 - accuracy: 0.9040 - val_loss: 0.2567 - val_accuracy: 0.9384 Epoch 57/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3300 - accuracy: 0.9058 - val_loss: 0.2722 - val_accuracy: 0.9203 Epoch 58/60 35/35 [==============================] - 0s 9ms/step - loss: 0.3204 - accuracy: 0.9013 - val_loss: 0.2523 - val_accuracy: 0.9529 Epoch 59/60 35/35 [==============================] - 0s 8ms/step - loss: 0.3533 - accuracy: 0.8940 - val_loss: 0.2633 - val_accuracy: 0.9457 Epoch 60/60 35/35 [==============================] - 0s 8ms/step - loss: 0.3433 - accuracy: 0.9067 - val_loss: 0.2568 - val_accuracy: 0.9493
<keras.callbacks.History at 0x1991de3e160>
score = imp_cnn_model.evaluate(X_train_mix, y_train_mix, verbose=0)
print("Training Accuracy: ", score[1])
score = imp_cnn_model.evaluate(X_test_mix, y_test_mix, verbose=0)
print("Testing Accuracy: ", score[1])
Training Accuracy: 0.960869550704956 Testing Accuracy: 0.7708333134651184
The system can classify a given speech as faked or otherwise on the fly given no information about the speaker.
we were able to deduce that the nature of spoken language may be important as we got less that 60% accuracy in classifying via a network trained by English-only samples.
Accuracy improved when the network was trained on mix data, Arabic constituted approximate 18 − −20% of the dataset. The performance of our model is proven by its accuracy which was at least 94%.
Further attempts were made to improve the performance by using more sophisticated networks such as CNN.
Dropout, regularization and early stopping techniques were applied to reduce overfitting.